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ABSTRACT 


This report summarizes results of a Study of two problems 
faced by the intelligence community: (1) lack of meaningful 
feedback from users on levels of satisfaction so as to cause 
product improvement and (2) nonexistence of qualitative or quan- 
titative data justifying intelligence programs for use in making 
budget decisions. It builds on previous work which identified 
problems in S&TI support to R&D and devised an improved system 
for providing that support. The study demonstrates wider applic- 
ability of the support system and shows that the component for 
evaluation of intelligence products could be used to develop data 
for decisionmaking by intelligence Managers, particularly in the 
product improvement, source assessment, and budget review 
processes. 
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EXECUTIVE SUMMARY 


PURPOSE AND BACKGROUND 


ds; This report documents results of a study undertaken 
to (a) surface problems in evaluating intelligence products 

and (b) devise solutions to those problems. The study builds 
on previous work that identified difficulties in providing 
scientific and technical intelligence to the materiel acquisi- 
tion community and outlined a rational system for providing 
that intelligence. 


COMMUNITY PROBLEMS 


Ze There are two principal problems with regard to 
evaluation of intelligence products, viz: 


a. There are no community-wide arrangements 
to provide feedback from users so as to 
cause product improvement. As a result, 
producers evaluate their own products. 


b. Intelligence community budget decisions 
must be made in the absence of adequate 
intelligence program justification. 
There is no means of developing documen- 
tation on user requirements or user 


Satisfaction. 
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SELECTED FINDINGS 


3. The principal study findings are contained in the 
following summary sentences. Amplifying portions of the report 
are shown in parentheses. 


e There is no community-wide system for 
intelligence product evaluation. Nor 
does the CIA, DIA, or any service have 
an agency or departmental system. 
(Paragraphs 2.6-2.11) 


e Producers of intelligence products 
do not know users' views. Producers 
self-evaluate by default. Products 
are not tailored to user needs. Users 
view the intelligence community as 
unresponsive. (Paragraph 2.13) 


e Systematic evaluation of intelligence 
products using customer input has been 
demonstrated and is feasible. Chapter 
II describes a workable system that will 
result in user satisfaction through 
product improvement. It requires no 
additions to existing organizations or 
personnel. It is not a demanding sys- 
tem, since it is highly selective from 
the viewpoint of the user and requires 
action only on those intelligence prod- 
ucts of unusual value to a given user. 
(Paragraphs 2.12-2.33, Figures 2.1 and 
2.2, Table 2.1) 
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e Managers of major programs in one ser- 
vice often are vitally dependent on 
intelligence from other services and 
agencies. Therefore, evaluation should 
be a community effort rather than a 
compartmented activity. (Paragraph 
3.14 and Table 3.3) 


@ A rational product evaluation system 
would generate the management infor- 
mation needed at all levels of the 
intelligence community for product 
improvement and budget review. Deci- 
sions based on this information could 
ensure that the NFIB budget was being 
spent to support valid needs of users. 
The information would also provide 
documented justification of intelli- 
gence programs for oversight review. 
(Paragraphs 2.17 and 3.15) 


RECOMMENDATION 
ig CU 
4. That the DCI sponsor a conference of representatives = f(x: 
from CIA, DIA, and the service intelligence agencies to work 
out procedures for instituting a community-wide product evalu- 
ation system. The system would make use of existing organiza- 


tions and personnel. 
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